Information retrieval in multimedia databases using relevance feedback algorithms. Applying logistic regression to relevance feedback in image retrieval systems

This master tesis deals with the problem of image retrieval from large image databases. A particularly interesting problem is the retrieval of all images which are similar to one in the user's mind, taking into account his/her feedback which is expressed as positive or negative preferences for the images that the system progressively shows during the search. Here, a novel algorithm is presented for the incorporation of user preferences in an image retrieval system based exclusively on the visual content of the image, which is stored as a vector of low-level features. The algorithm considers the probability of an image belonging to the set of those sought by the user, and models the logit of this probability as the output of a linear model whose inputs are the low level image features. The image database is ranked by the output of the model and shown to the user, who selects a few positive and negative samples, repeating the process in an iterative way until he/she is satisfied. The problem of the small sample size with respect to the number of features is solved by adjusting several partial linear models and combining their relevance probabilities by means of an ordered weighted averaged (OWA) operator. Experiments were made with 40 users and they exhibited good performance in finding a target image (4 iterations on average) in a database of about 4700 imagesZuccarello, PD. (2007). Information retrieval in multimedia databases using relevance feedback algorithms. Applying logistic regression to relevance feedback in image retrieval systems. http://hdl.handle.net/10251/12196Archivo delegad

Development and implementation of a selective change-driven vision sensor for high speed movement analysis

Un sistema de vision artificial esta compuesto, en su forma más basica, por un sensor VLSI, habitualmente fabricado en tecnología CMOS o CCD, y una etapa de procesado. En la gran mayoría de los sistemas de visión artificial implementados hoy en día la etapa sensora del sistema consiste en un sensor de imágenes tradicional. Este tipo de sensores trabajan bajo unos principios muy simples y conocidos: el nivel de iluminación del entorno es muestreado y transmitido a intervalos de tiempo regulares; y todos los píxeles de la matriz, sin excepción, son transmitidos secuencialmente y en orden. Esto es así aunque no se hayan producido cambios en la escena bajo observación. Esto implica que una gran parte de la información que se genera y transmite puede ser considerada como redundante. En muchos casos esta estrategia es la más adecuada. Algunos ejemplos de ello son los escáneres, los sistemas de captura de imágenes para diagnóstico médico o los sistemas de video para entretenimiento. Todas estas aplicaciones necesitan la mayor cantidad de información posible sobre el entorno, aunque este no cambie o muestre variaciones muy pequeñas en intervalos de tiempo largos. Para otro tipo de aplicaciones, como los sistemas de visión artificial o las redes de sensores inalámbricas, la gran cantidad de información redundante que genera y transmite un sensor tradicional de imágenes puede convertirse en una limitación para la implementación de sistemas en muchos entornos reales. Muchos sistemas de visión biológicos trabajan de manera completamente distinta a los sensores de captura de imágenes tradicionales. Una de sus principales características es que las celdas sensibles (el equivalente de los píxeles en tecnología de silicio) reaccionan de manera independiente y asíncrona a los cambios de iluminación. Tomando como punto de partida los trabajos de C.Mead y M.Mahowald realizados a finales de los años 80, las últimas dos décadas han presenciado avances muy significativos en el diseño de sensores de visión, todos estos fundamentalmente orientados a transmitir y procesar solo la información considerada importante o relevante dentro de la escena bajo análisis. La mayor parte de estos diseños han tomado, en mayor o menor medida, el funcionamiento del sistema biológico de visión como base de sus desarrollos. El objetivo de muchos de los trabajos realizados en este área es imitar de la mejor manera posible, y mediante las más avanzadas tecnologías de silicio, el comportamiento de los sistemas biológicos en sus facetas visual, auditiva y cognitiva. Otros trabajos han seguido otra filosofía, tomando la biología como fuente de inspiración, pero no como un objetivo en sí mismo. La estrategia de visión selectiva guiada por cambios (SCD por sus siglas en inglés) pertenece a este último grupo. Orientada a la detección y análisis de objetos moviéndose a alta velocidad, la estrategia SCD asume que solo un parte de la imagen muestra cambios entre dos frames consecutivos, mientras que la mayor parte de los píxeles permanecen igual. Esta hipótesis cobra especial sentido cuando se capturan frames a alta velocidad. Teniendo en cuenta que muchos de los píxeles de una determinada imagen no han cambiado respecto de sus valores en las imágenes anteriores de la secuencia, los algoritmos de procesado pueden utilizar la información ya almacenada para realizar sus cálculos. Es decir, que esta información redundante podría no transmitirse. Se podría incluso considerar que los píxeles de la matriz que muestran cambios pequeños, tendrán poco impacto en los resultados de los algoritmos. En la estrategia SCD estas hipótesis son trabajadas de forma tal que se consigue reducir sustancialmente la cantidad de información transmitida por el sensor, y por lo tanto la cantidad de información procesada fuera del mismo. En la estrategia SCD ya no se trabaja con imágenes de forma estática, sino que la información es transportada y transmitida en la forma de un flujo de píxeles. Estos píxeles son seleccionados de forma tal que contengan solo la información con cambios temporales relevantes dentro de la escena bajo análisis. Bajo estas nuevas condiciones, sería necesario el rediseño de muchos de los algoritmos de visión tradicionales, ya que estos trabajan en base a una secuencia de imágenes estáticas transmitidas a intrevalos de tiempo regulares. El paradigma de procesado por flujo de datos (data-flow processing) parace ajustarse de manera más adecuada a esta nueva forma de trabajo. En esta tesis, se presenta el primer sensor de visión basado en los principios SCD. Dicho sensor consiste en una matriz de 32x32 píxeles fabricada en tecnología CMOS de 350 nm. La mayor dificultad del diseño microelectrónico presentado en esta tesis es el diseño del bloque que selecciona el pixel de mayor cambio entre todos los de la matriz. Este problema ha sido resuelto mediante un circuito winner-takes-all (WTA). La propuesta de un circuito digital para la selección de un unico ganador en una matriz WTA compuesta por una gran cantidad de celdas es uno de los aportes originales de esta tesis. El sensor fue empotrado en un sistema de visión artifical portátil basado en un microcontrolador de 32 bits trabajando a 80 MHz. Este sistema ha sido utilizado para la implementación de un algoritmo de seguimiento de objetos así como para la caracterización misma del sensor. Con la experimentación presentada en esta tesis se demuestra como una sistema SCD simple y portátil, como el desarrollado aquí, se puede hacer el seguimiento de un objeto en movimiento con la resolución temporal de una cámara de alta velocidad trabajando a 2000 frames por segundo, pero utilizando solo el ancho de banda que utilizaría una cámara estándar de baja velocidad trabajando a 25 frames por segundo. Esto demuestra claramente que la utilización de la estrategia SCD implica una reducción substancial en los requisitos de ancho de banda y potencia de cálculo del sistema.An artificial vision system is basically composed of a sensor, usually in VLSI CMOS or CCD technology, and a processing stage. Nowadays, in the vast majority of real-world implementations, the sensing part of the system is a traditional frame-based imager. These types of image sensors work under some very well known principles: the illumination level of the surrounding environment is sampled and transmitted at regular time intervals, even if no new relevant information is produced in the scene under analysis. A traditional frame-based image sensor is usually not able to evaluate if the information coming from a certain pixel is relevant or irrelevant. Since they do not perform any kind of analysis of the information being captured, the illumination level of all the pixels in the sensing matrix must be transmitted to be analyzed and processed at the processing stage. Many times, a huge amount of redundant non-relevant information is transmitted. The consequences of this are that valuable resources such as bandwidth and processing power are wasted. Furthermore, depending on the particular context and hardware configuration, the processing hardware may not even be able to cope with all the generated data. Many of these problems can be overcome with the design of new sensing and readout strategies focused on the selection of relevant changing information. Over the last decade many relevant improvements have been achieved in this direction. Taking the biological vision system as a general guide and inspiration, an increasing number of very-large scale of integration (VLSI) vision sensors have been, and are being designed where the sparcity, asynchrony and event-driven generation of the information coming from the visual field is taken into account. It is within this framework that Selective Change-Driven Vision (SCD) emerges as an innovative and original proposal. SCD Vision relies on the idea that a pixel showing a large change in intensity is an indicator of fast movements, and object edges around it. An SCD sensor is frame-based in the sense that successive frames are captured at a very high rate, but pixel readout is performed in an entirely different manner. The pixels are read out in order of relevance. The larger the change in illumination, the more relevant the pixel is considered to be. Not all the pixels in the sensing matrix need to be transmitted. As the pixels showing relevant changing information are transmitted first, a small subset of pixels might be read out, these being the ones conveying the most important information of the scene under analysis. In this thesis, the first VLSI CMOS vision sensor following SCD principles is presented. A 32x32 pixel matrix was implemented and fabricated in 0.35 μm 4-metal 2-poly silicon technology. The most challenging part of this microelectronic design was the decision block, where the pixels undergoing the largest changes in the sensing matrix are selected. This problem was solved by means of a winner-takes-all (WTA) circuit. A large WTA network together with a proposal for single winner selection was designed, implemented and its behaviour characterized. The designed sensor was embedded into a small, but powerful artificial vision system based on a 32-bit microcontroller. This system was used to implement tracking algorithms as well as to characterize the main basic features of the sensor. The experimentation carried out in this thesis shows how a simple SCD system based on our SCD sensor is able to track fast moving objects with just the bandwidth requirements of a low speed 25 fps standard camera, but with the time resolution and performance of a high-speed camera working at 2000 fps. This clearly demonstrates that bandwidth and processing requirements are substantially reduced when SCD hardware is used

Taking Advantage of Selective Change Driven Processing for 3D Scanning

This article deals with the application of the principles of SCD (Selective Change Driven) vision to 3D laser scanning. Two experimental sets have been implemented: one with a classical CMOS (Complementary Metal-Oxide Semiconductor) sensor, and the other one with a recently developed CMOS SCD sensor for comparative purposes, both using the technique known as Active Triangulation. An SCD sensor only delivers the pixels that have changed most, ordered by the magnitude of their change since their last readout. The 3D scanning method is based on the systematic search through the entire image to detect pixels that exceed a certain threshold, showing the SCD approach to be ideal for this application. Several experiments for both capturing strategies have been performed to try to find the limitations in high speed acquisition/processing. The classical approach is limited by the sequential array acquisition, as predicted by the Nyquist - Shannon sampling theorem, and this has been experimentally demonstrated in the case of a rotating helix. These limitations are overcome by the SCD 3D scanning prototype achieving a significantly higher performance. The aim of this article is to compare both capturing strategies in terms of performance in the time and frequency domains, so they share all the static characteristics including resolution, 3D scanning method, etc., thus yielding the same 3D reconstruction in static scenes

An Open-set Recognition and Few-Shot Learning Dataset for Audio Event Classification in Domestic Environments

The problem of training a deep neural network with a small set of positive samples is known as few-shot learning (FSL). It is widely known that traditional deep learning (DL) algorithms usually show very good performance when trained with large datasets. However, in many applications, it is not possible to obtain such a high number of samples. In the image domain, typical FSL applications are those related to face recognition. In the audio domain, music fraud or speaker recognition can be clearly benefited from FSL methods. This paper deals with the application of FSL to the detection of specific and intentional acoustic events given by different types of sound alarms, such as door bells or fire alarms, using a limited number of samples. These sounds typically occur in domestic environments where many events corresponding to a wide variety of sound classes take place. Therefore, the detection of such alarms in a practical scenario can be considered an open-set recognition (OSR) problem. To address the lack of a dedicated public dataset for audio FSL, researchers usually make modifications on other available datasets. This paper is aimed at providing the audio recognition community with a carefully annotated dataset for FSL and OSR comprised of 1360 clips from 34 classes divided into pattern sounds and unwanted sounds. To facilitate and promote research in this area, results with two baseline systems (one trained from scratch and another based on transfer learning), are presented.Comment: To be submitted to Expert System with Application

Selective Change Driven Imaging: A Biomimetic Visual Sensing Strategy

Selective Change Driven (SCD) Vision is a biologically inspired strategy for acquiring, transmitting and processing images that significantly speeds up image sensing. SCD vision is based on a new CMOS image sensor which delivers, ordered by the absolute magnitude of its change, the pixels that have changed after the last time they were read out. Moreover, the traditional full frame processing hardware and programming methodology has to be changed, as a part of this biomimetic approach, to a new processing paradigm based on pixel processing in a data flow manner, instead of full frame image processing

Measurement of the cosmic ray spectrum above 4×10184{\times}10^{18} eV using inclined events detected with the Pierre Auger Observatory

A measurement of the cosmic-ray spectrum for energies exceeding $4{\times}10^{18}$ eV is presented, which is based on the analysis of showers with zenith angles greater than $60^{\circ}$ detected with the Pierre Auger Observatory between 1 January 2004 and 31 December 2013. The measured spectrum confirms a flux suppression at the highest energies. Above $5.3{\times}10^{18}$ eV, the "ankle", the flux can be described by a power law $E^{-\gamma}$ with index $\gamma=2.70 \pm 0.02 \,\text{(stat)} \pm 0.1\,\text{(sys)}$ followed by a smooth suppression region. For the energy ($E_\text{s}$) at which the spectral flux has fallen to one-half of its extrapolated value in the absence of suppression, we find $E_\text{s}=(5.12\pm0.25\,\text{(stat)}^{+1.0}_{-1.2}\,\text{(sys)}){\times}10^{19}$ eV.Comment: Replaced with published version. Added journal reference and DO